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REMOTE SURGERY SUPPORT SYSTEM 
CROSS-REFERENCE TO RELATED APPLICATIONS 

[0001] This is a divisional of U.S. Patent Application Serial No. 09/503,520, 
filed February 14, 2000 in the name of Akinobu UCHIKUBO and entitled REMOTE 
SURGERY SUPPORT SYSTEM. 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

[0002] The present invention relates to a remote surgery support system for 
remotely supporting surgery. 

2. Description of the Related Art 

[0003] Normally, a surgeon performs surgery on a patient in an operating room. 
Sometimes the surgeon may have to carry out a surgical procedure with which the 
surgeon has little experience. A support system for helping the surgeon appropriately 
perform surgery on a patient is presumably such that the surgeon in the operating 
room can perform surgery while being supported by a remotely supporting surgeon. 
Specifically, the surgeon in the operating room communicates with the remotely 
supporting surgeon in a remote place over a public line during surgery. 
[0004] Related arts concerning the remote surgery support system for remotely 
supporting surgery include Japanese Patent Application No. 10-318019. 
[0005] According to the related art, when surgery is performed by controlling 
the movement of a surgical instrument using a manipulator, a surgeon in an operating 
room is informed of the manipulation expressed by a manipulation signal produced 
when a remotely supporting surgeon controls the surgical instrument. Consequently, 
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the surgeon in the operating room can smoothly carry out a surgical procedure with 
the help of the remotely supporting surgeon. 

[0006] However, according to the related art, the remotely supporting surgeon 
cannot monitor the state of the surgical instrument in the operating room. For 
instructing the way of manipulating the surgical instrument from a remote control 
room to the operating room, the remotely supporting surgeon must ask verbally as to 
the state of the surgical instrument from time to time. A surgical procedure is 
therefore suspended frequently. Consequently, there is an increasing demand for an 
environment in which surgery can be performed more smoothly. 
[0007] Moreover, the remotely supporting surgeon has no means for checking 
patient data. Patient information must therefore be sent to the remotely supporting 
surgeon. In case of emergency surgery, it is impossible to make patient information 
timely available in a remote place. 

SUMMARY OF THE INVENTION 

[0008] An object of the present invention is to provide a remote surgery support 
system making it possible to readily check the state of a surgical instrument and/of 
patient information in a remote place over a communication line. 
[0009] Another object of the present invention is to provide a remote surgery 
support system making it possible to give instructions on surgery while readily 
checking the state of a surgical instrument and/or patient information in a remote 
place over a communication line. 

[0010] Still another object of the present invention is to provide a surgical 
information display method making it possible to observe endoscopic images and 
readily check the state of a surgical instrument and/or patient information. 
[0011] A remote surgery support system consists mainly of: 
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[0012] a first control means installed in an operating room for controlling a 
surgical instrument; 

[0013] an input means for use in inputting control information, with which the 
surgical instrument is controlled, to the first control means; 
[0014] a first signal transmission apparatus for receiving information, which 
indicates the controlled state of the surgical instrument, from the first control means, 
and converting the information into a transmissible signal; 

[0015] a communication line over which the transmissible signal converted by 
the first signal transmission apparatus to a control room in a remote place; 
[0016] a second signal transmission apparatus for receiving the transmissible 
signal from the first signal transmission apparatus over the communication line, and 
converting the signal into a signal corresponding to the information; 
[0017] a second control means for receiving the signal from the second signal 
transmission apparatus; and 

[0018] a display device for displaying an output of the second control means. 
[0019] Consequently, surgeon staying in the remote place can monitor the state 
of the surgical instrument. 

[0020] The remote surgery support system further includes: 

[0021] a second input means for supplying support information to the second 

control means; and 

[0022] a second display device for inputting the support information from the 
second control means via the second signal transmission apparatus and the first 
signal transmission apparatus, and displaying the information. 
[0023] Consequently, the surgeon in the remote place can input support 
information concerning a procedure at the second input means to the second control 
means. The input support information is displayed on the second display device via 
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the second signal transmission apparatus and the first signal transmission apparatus. 
The surgeon in the operating room can carry out the procedure. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0024] The invention is described in detail below with reference to the 
following figures in which similar reference characters denote corresponding features 
consistently throughout, wherein: 

[0025] FIG. 1 to FIG. 6 relate to the first embodiment of the present invention; 
[0026] FIG. 1 is a schematic view of a remote surgery support system in 
accordance with the first embodiment; 

[0027] FIG. 2 is a schematic view of the remote surgery support system shown 
in FIG. 1; 

[0028] FIG. 3 is a schematic view of a controller in a surgery system; 
[0029] FIG. 4 is a schematic view of a second controller in a remote support 
system; 

[0030] FIG. 5 is a display view of a display screen of a display device; 

[0031] FIG. 6 is a flowchart of a sequence of information display involving the 

surgery system and remote support system; 

[0032] FIG. 7 is a schematic view of a remote surgery support system in 

accordance with the second embodiment of the present invention; 

[0033] FIG. 8 is a display view of a display screen of a display device in 

accordance with the third embodiment of the present invention; 

[0034] FIG. 9 and FIG. 10 relate to the fourth embodiment of the present 

invention; 

[0035] FIG. 9 is a schematic view of a remote surgery support system in 
accordance with the fourth embodiment; 
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[0036] FIG. 10 is a schematic view of the remote surgery support system shown 
in FIG. 9; 

[0037] FIG. 1 1 is a schematic view of a remote surgery support system in 

accordance with the fifth embodiment of the present invention; 

[0038] FIG. 12 is a schematic view of a remote surgery support system in 

accordance with the sixth embodiment of the present invention; 

[0039] FIG. 13 is a schematic view of a remote surgery support system in 

accordance with the seventh embodiment of the present invention; 

[0040] FIG. 14 to FIG. 18 relate to the eighth embodiment of the present 

invention; 

[0041] FIG. 14 is a schematic view of a medical information processing system; 

[0042] FIG. 15 is a schematic view of an information accumulation system; 

[0043] FIG. 16 is a schematic view of a control circuit; 

[0044] FIG. 17 is a display view of a medical information display screen; 

[0045] FIG. 18 is a print out view of a clinical recording printout; 

[0046] FIG. 19 to FIG. 22 relate to the ninth embodiment of the present 

invention; 

[0047] FIG. 19 is a schematic view of a medical information processing system; 

[0048] FIG. 20 is a schematic view of a control apparatus for producing clinical 
recordings; 

[0049] FIG. 21 is a display view of a displayed integrated information screen; 
and 

[0050] FIG. 22 is a printout view of the a printed clinical recording. 
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DESCRIPTION OF PREFERRED EMBODIMENTS 

[0051] Referring to the drawings, embodiments of the present invention will be 
sequentially described below. To begin with, the first embodiment will be described 
with reference to FIG. 1 to FIG. 6. 

[0052] The present embodiment is a remote surgery support system. An 
operating room and a remote control room in a remote place are liked with a 
communication line. A surgeon in the operating room receives support information 
from the remote control room and performs surgery. 

[0053] As shown in FIG. 1 or FIG. 2, a remote surgery support system 1 in 
accordance with the first embodiment of the present invention has a surgery system 2 
connected to a remote support system 3. The support system 3 is connected remotely 
to the surgery system 2, over a public line 4. The surgery system 2 and remote 
support system 3 are installed in an operating room 5 and a remote control room on 
remote support system room 6, respectively. 

[0054] The surgery system 2 installed in the operating room 5 has an 
endoscopic imaging apparatus 10 and a surgical apparatus or surgical instrument 20. 
The endoscopic imaging apparatus 10 is used to observe an intracavitary region of a 
patient 7. The surgical apparatus or surgical instrument 20 is used to perform surgery 
on the patient 7 for curing a lesion under observation through the endoscopic 
imaging apparatus 10. 

[0055] To be more specific, as shown in FIG. 2, a surgical instrument body 8 
and an optical endoscope 9 are advanced into, for example, the abdominal cavity of 
the patient 7. The surgical instrument body 8 is used to perform surgery for curing a 
lesion. The optical endoscope 9 is used to observe the progress of surgery performed 
using the surgical instrument body 8. 

[0056] The surgical instrument body 8 is an instrument for incising or 
coagulating a lesion using, for example, electric cautery. The surgical instrument 
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body 8 is connected to a power supply unit 1 1 . The power supply unit 1 1 supplies 
driving power to the surgical instrument body 8 over a cord or the like. The power 
supply unit 1 1 has the ability to permit variance of the setting of an output value or 
an output waveform of electric energy according to whether an incision mode or a 
coagulation mode is selected. The surgical instrument body 8 and the power supply 
unit 1 1, or control unit in some instruments, constitute the surgical instrument 20. 
[0057] The endoscope 9 is, for example, a rigid endoscope having a rigid 
insertion unit 12. A TV camera 15, having, for example, a charge couple device 
(CCD) 14 as an imaging device incorporated therein, is mounted on an eyepiece unit 
13 located at the back end of the insertion unit 12, whereby a means for producing 
endoscopic images is realized. 

[0058] A light guide cable 16 extending from the endoscope 9 is linked to a 
light source apparatus 17. Light emanating from a lamp (not shown) in the light 
source apparatus 17 is propagated over a light guide contained in the light guide 
cable 16 and a light guide through the endoscope 9. The light is emitted through the 
end surfaces of the light guides fixed in an illumination window formed at the distal 
end of the insertion unit 12. An object, such as an intracavitary organ, is thus 
illuminated. 

[0059] An objective lens (not shown) is fixed in an observation window 
adjoining the illumination window, and forms an optical image of an object. The 
optical image is propagated backwardly along, for example, a system of relay lenses 
serving as an optical image propagating means located in the insertion unit 12. The 
optical image can be viewed in enlarged proportions through an eyepiece (not shown) 
in the eyepiece unit 13. 

[0060] The propagated optical image is projected on the CCD 14 via an image 
formation lens 18 in the TV camera 15. The TV camera 15 is mounted on the 
eyepiece unit 13 so that it can be dismounted freely. The CCD 14 is connected to a 
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camera control unit (hereinafter CCU) over a signal cable 19. A signal 
photoelectrically converted by the CCD 14 is processed by a video processing circuit 
21a in the CCU 21. The video processing circuit 21a produces and outputs a 
standard video signal to a first monitor 22 via a mixer 21b. 

[0061] The video signal sent from the CCU 14 is output to the first monitor 22. 
An endoscopic image representing the intracavitary organ projected on the CCD 14 
and the distal part of the surgical instrument body 8 used for surgery is displayed. 
[0062] The CCU 21 and power supply unit 1 1 are connected to a first controller 
24 for controlling the CCU 21 and the power supply unit 11. 
[0063] FIG. 3 shows the configuration of the controller 24. The controller 24 
consists mainly of a central processing unit (CPU) 35, a hard disk drive (HDD) 36, a 
memory 37, and an input/output (I/O) interface 38a. The CPU 35 controls the 
controller 24. The HDD 36 is used to store operation programs according to which 
the CPU 35 operates. The memory 37 is used to temporarily store an image and as a 
work area. The I/O interface 38a allows input or output of control signals or the like 
via a signal transmission unit 27. The CPU 35, HDD 36, memory 37, and I/O 
interface 38a are interconnected over a bus 39. 

[0064] A touch panel 25 used to input instructions, a magnetic card reader 26 
for inputting patient data or the like, and a keyboard 29 are interconnected over the 
bus 39 via interfaces 25a, 26a, and 29a respectively. 

[0065] Interfaces 38b and 38c output control signals and patient data to the 
video processing circuit 21a and mixer 21b in the CCU 21 . An interface 38d outputs 
the control signals to the power supply unit 1 1 in the treatment instrument 20. 
[0066] Assuming that a command or the like is input for changing the tone of 
endoscopic images using, for example, the touch panel 25, a control signal used to 
change the tone is output from the controller 24 to a tone setting circuit in the video 
processing circuit 21a in the CCU 21 . Tone change is thus controlled. The power 
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supply unit 1 1 in the surgical instrument 20 is controlled in order to control output of 
electric energy to the surgical instrument body 8. 

[0067] When the surgical instrument body 8 is, for example, an electric cautery 
instrument, setting of the output level of electrical energy to be supplied to the 
instrument body 8 for incision or coagulation can be controlled remotely with 
application of a control signal to the power supply unit 1 1 . When the treatment 
instrument 20 is a pneumoperitoneum unit, a set value of pressure can be varied. 
[0068] The magnetic card reader 26 is used to read patient data recorded on a 
magnetic card. The patient data is input to the controller 24, and output to the mixer 
21b in the CCU 21 via the controller 24. The patient data can thus be superimposed 
on an endoscopic image. 

[0069] The CCU 21 and the controller 24 installed in the operating room 5 are 
connected to the first signal transmission apparatus or signal conversion apparatus 
27. A video signal expressing endoscopic images is output from the CCU 21 to the 
first monitor 22. The video signal is converted into a signal, which is transmissible 
over the public line 4, such as an integrated services digital network (ISDN), using 
the signal transmission apparatus 27. The resultant signal is then transmitted to a 
second signal transmission apparatus or signal conversion apparatus 28 in the remote 
control room 6 over the public line 4. 

[0070] A control signal sent from the controller 24 or patient data is converted 
into a signal, which is transmissible over the public line 4, using the signal 
transmission apparatus 27. The resultant signal is then transmitted to the signal 
transmission apparatus 28 in the remote control room 6 over the public line 4. 
[0071] A signal sent from the signal transmission apparatus 28 in the remote 
control room 6 to the signal transmission apparatus 27 over the public line 4 is 
converted or demodulated into a video signal corresponding to a signal that has not 
been converted by the signal transmission apparatus 28. The resultant signal is then 
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output to the second monitor 29 connected to the signal transmission apparatus 27. 
Consequently, image information sent from the signal transmission apparatus 28 can 
be displayed on the second monitor 29. 

[0072] The signal transmission apparatuses 27 and 28 convert an input signal 
into a transmissible signal, and place the transmissible signal on the public line 4. 
The signal transmission apparatuses 27 and 28 have the ability to convert or 
demodulate) an input signal, which has been transmitted over the public line 4, into a 
signal corresponding to a signal that has not been converted or modulated. In short, 
the signal transmission apparatuses 27 and 28 provide interface means permitting 
bi-directional communication over the public line 4. 

[0073] The keyboard 29 used to input character data or the like is connected to 
the controller 24. using the keyboard 29, character data can be transmitted to a 
remotely supporting surgeon or the signal transmission apparatus 28 via the 
controller 24. 

[0074] The remote support system 3 in the remote control room 6 has a third 
monitor 31 connected to the signal transmission apparatus 28. Endoscopic images 
sent from, for example, the CCU 21 in the operating room 5 are displayed on the 
third monitor 31. 

[0075] The signal transmission apparatus 28 is connected to a second controller 
32. A display device 33 serving as a fourth monitor is connected to the controller 32. 
An input means used to enter support information, for example, a touch panel or a 
keyboard 34, is connected to the controller 32. A pointing device (not shown), such 
as a mouse, is also connected to the controller 32. 

[0076] The controller 32 captures an endoscopic image, which is sent from the 
CCU 21 in the operating room 5 via the signal transmission apparatuses 27 and 28, 
as a still image. Patient information sent from the first controller 24 via the signal 
transmission apparatuses 27 and 28 is input to the controller 32. The controller 32 
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superimposes the patient information on the endoscopic image and displays the 
resultant endoscopic image on the display device 33. The keyboard 34 or any other 
input means is used to enter an indication, for example, a mark indicating a position 
to be resected or the position of an artery to which attention must be given during 
resection. The indication is displayed as an overlay on the display device 33. This is 
intended to provide a surgeon in the operating room 5 with support information such 
as instruction information needed for surgery. 

[0077] Image information or the like to be displayed as an overlay on the 
display device 33 is transmitted to the second monitor 29 in the operating room 5 via 
the signal transmission apparatuses 28 and 27. The contents of the image information 
are then displayed on the display screen of the second monitor 29. A surgeon in the 
operating room 5 performs surgery while viewing the image that is displayed on the 
second monitor 29 and that has the support information entered by the remotely 
supporting surgeon overlaid. Thus, surgery can be performed properly. 
[0078] FIG. 4 shows the configuration of the controller 32. The controller 32 
consists mainly of a central processing unit (CPU) 40, a hard disk drive (HDD) 41, a 
memory 42, an input/output (I/O) interface 43 f a video capture control unit 44, and a 
keyboard interface 45. The CPU 40 controls the controller 32. The HDD 41 is used 
to store operation programs according to which the CPU 40 operates, and images. 
The memory 42 is used to temporarily store an image and as a work area. The I/O 
interface 43 permits input or output of control signals or the like via the signal 
transmission apparatus 28. The video capture control unit 44 captures a video signal 
and performs superimposition. The keyboard interface 45 is connected to, for 
example, the keyboard 34. The CPU 40, HDD 41, memory 42, I/O interface 43, 
capture control unit 44, and keyboard interface 45 are interconnected over a bus 49a. 
[0079] Communication of a control signal or the like sent from the second 
signal transmission apparatus 28 is achieved via the I/O interface 43. The operation 
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programs according to which the controller 32 operates are stored in the HDD 41 . 
For example, controlling the movement of the surgical instrument 20 is established at 
the touch panel 25 in the operating room 5 via the first controller 24. The contents of 
the control are stored in the memory 42 through the I/O interface 43 in the controller 
32 via the signal transmission apparatuses 27 and 28. Patient information is stored in 
the memory 43 through the I/O interface 43 in the controller 32. 
[0080] The video capture control unit 44 is connected to the signal transmission 
apparatus 28. The video capture control unit has an A/D converter 46 for digitizing 
an input video signal and a D/A converter 47 for converting the video signal into an 
analog form and outputting the analog video signal. 

[0081] The A/D converter 46 and D/A converter 47 are connected to an overlay 
control unit 48 for controlling an overlay. The overlay control unit 48 has a video 
memory 48a in which endoscopic image data or the like are stored. A video signal 
converted by the A/D converter 46 is stored in the video memory 48a. Otherwise, a 
video signal stored in the video memory 48a is output to the display device 33 via the 
D/A converter 47, and then output to the signal transmission apparatus 28. 
[0082] The video capture control unit 44 is connected to a control circuit 49 for 
controlling display of an overlay and transmitting or receiving data. The control 
circuit 49 is connected to a bus 49a. 

[0083] According to the present embodiment, communication of images by the 
signal transmission apparatus 28 is achieved via the A/D converter 46 and D/A 
converter 47 constituting the video capture control unit 44. A video signal input from 
the A/D converter 46 is converted into image data by the overlay control unit 48 
under the control of the control circuit 49. 

[0084] An output of the overlay control unit 48 is transmitted to the signal 
transmission apparatus 28 via the D/A converter 47. The CPU 40 controls 
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communication of the signal transmission apparatus 28 with the controller 32 
according to a program stored in the HDD 41 . 

[0085] An image captured using the video capture control unit 44 can be stored 
in the HDD 41. An image stored in the HDD 41 is selected according to an entry 
made by pressing keys of the keyboard 34 for selecting an image. The CPU 40 
outputs a reduced image or thumbnail image of the selected image to the video 
capture control unit 44. The reduced image signal may be superimposed on a video 
signal sent from the first signal transmission apparatus 27 using the overlay control 
unit 48. 

[0086] Images sent from the signal transmission apparatus 28 are converted by 
the A/D converter 46, and captured by the video capture control unit 44. The image 
as well as control signals and patient data input via the I/O interface 43 and control 
circuit 49 can be written in any area in a video memory 48a in the overlay control 
unit 48. 

[0087] A video signal transferred from the D/A converter 47 is also output to 
the display device 33. The display device 33 can display images like the ones shown 
in FIG. 5. 

[0088] A display area 33a on the display device 33 consists of a screen display 
area 50, a thumbnail display area 51, a state-of-surgical instrument display area 52, a 
patient information display area 53, and a comment display area 54. 
[0089] The thumbnail display area 5 1 is an area in which a plurality of images 
relevant to a selected item is displayed. For selecting a desired item, such as patient 
name, procedure name, or the like, an image selection button (not shown) appearing 
on a tool bar displayed in the display area 33a is manipulated in order to open an 
image window. 
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[0090] Displayed in the screen display area 50 are images carried by a video 
signal sent from the CCU 21 forming an endoscopic imaging means and an image 
selected from the thumbnail display area 51. 

[0091] Displayed in the thumbnail display area 5 1 are reduced images of image 
data accumulated in the second controller 32, and a reduced image (thumbnail 
image) of a still image produced using a video signal that expresses a motion picture 
or endoscopic images and that is sent from the CCU 21 . 
[0092] Displayed in the state-of-surgical instrument display area 52 are the 
states of the surgical instrument body 8 and CCU 21 transmitted from the first 
controller 24. 

[0093] Displayed in the patient information display area 53 is patient 
information sent from the first controller 24. 

[0094] A comment transmitted from the surgery system 2 is displayed in the 

comment display area 54 interposed between the state-of-surgical instrument display 

area 52 and patient information display area 53 shown in FIG. 5. 

[0095] The standard H320 and others have been established as a standard for 

transmitting images, voice, or data over a public line. The standard H320 may be 

regarded as a standard stipulating transmission of images, voice, or signals over the 

ISDn. 

[0096] Operations to be undertaken by the present embodiment having the 
foregoing components will be described below. As shown in FIG. 1 or FIG. 2, the 
surgery system 2 and remote support system 3 are linked by the public line 4, such as 
the ISDN. The power supplies of the surgery system 2 and remote support system 3 
are then turned on. 

[0097] Patient information, including the name of the patient 7, is input from 
the magnetic card reader 26 to the first controller 24. 
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[0098] The endoscope 9 is connected to the light source apparatus 1 7 over the 
light guide cable 16 so that light can be supplied through the endoscope 9. The TV 
camera 15 is mounted on the eyepiece unit 13 of the endoscope 9. The signal cable 
19 extending from the TV camera 15 is linked to the CCU 21, so that an endoscopic 
image projected on the CCD 14 can be displayed on the first monitor 22. 
[0099] A pneumoperitoneum unit is advanced into the abdominal cavity of the 
patient 7 by way of a trocar and cannula (not shown) whereby air is supplied to the 
abdominal cavity. A flow rate at which air is supplied to the abdominal cavity is 
entered at the keyboard 29 or the like. 

[0100] The insertion unit 12 of the endoscope 9 is advanced by way of the 
trocar and cannula, so that an image of a lesion in the abdominal cavity can be 
displayed on the first monitor 22. 

[0101] The surgical instrument body 8, such as an electric cautery instrument 
used for surgery, is advanced into the abdominal cavity of the patient 7 by way of the 
trocar and cannula. The output value of electrical energy to be supplied to the electric 
cautery instrument for incision or coagulation is entered at the keyboard 29 or the 
like. The contents of control to be given for surgery are thus entered. 
[0102] For changing the tone of endoscopic images displayed on the first 
monitor 22, a control signal is sent from the first controller 24 to the CCU 21 by 
manipulating the touch panel 25. Thus, the tone can be changed. 
[0103] A video signal output from the CCU 21 to the first monitor 22 and 
carrying endoscopic images is transmitted from the first signal transmission 
apparatus 27 to the second signal transmission apparatus 28 over the public line 4, 
and then displayed on the third monitor 3 1 . 

[0104] The video signal can be input to the second controller 32. When an 
endoscopic image displayed on the third monitor 3 1 properly represents a region to 
be resected during surgery, the remotely supporting surgeon gives instructions at the 
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keyboard 34. The endoscopic image, a still image, is then captured using the video 
capture control unit 44 in the controller 32. The captured endoscopic image is, as 
shown in FIG. 5, then displayed in the screen display area 50 on the display device 
33 connected to the controller 32. 

[0105] Patient information read by the magnetic card reader 26 is input from the 
first controller 24 to the controller 32 through the I/O interface 43 via the signal 
transmission apparatuses 27 and 28. The patient information is stored in, for 
example, the memory 42 in the controller 32, and, as shown in FIG. 5, displayed in 
the patient information display area 53 on the display device 33 all the time. 
[0106] The remotely supporting surgeon displays the endoscopic image sent 
from the operating room 5 with the patient information overlaid. When the surgical 
instrument body 8 is used to resect a lesion, the position of a region to be resected is 
marked on a still image. A mouse or the keyboard 34 (not shown) is used to enter a 
mark on the still image. If attention must be given to an artery located near the 
region to be resected, the artery is marked in a color different from the color of the 
mark indicating the position of the region to be resected. Support information is thus 
displayed. 

[0107] When an electric cautery instrument is used to incise or coagulate a 
lesion, the surgeon in the operating room 5 enters the set value of electric energy at 
the touch panel 25. The contents of control including the setting of electric energy 
output to the surgical instrument 20 are sent from the first controller 24 to the I/O 
interface 43 of the second controller 32, and stored in, for example, the memory 42. 
Control information or setting information concerning the surgical instrument 20 is, 
as shown in FIG. 5, displayed in the state-of- surgical instrument display area 52. 
[0108] Referring to FIG. 5, setting information of a pneumoperitoneum unit is 
also displayed in the state-of-surgical instrument display area 52. Thus, control 
information concerning a plurality of surgical instruments can be displayed. 
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[0109] When the surgeon in the operating room 5 changes the output value of 
electric energy supplied to an electric cautery at the touch panel 25, the changed 
contents are sent to the I/O interface 43 in the second controller 32 and displayed in 
the state-of-surgical instrument display area 52. The contents of control or setting 
given to the surgical instrument 20 are displayed on the display device nearly in real 
time. The remotely supporting surgeon can check the state of the surgical instrument 
20 nearly in real time. 

[01 10] When the surgeon in the operating room 5 has a comment about the 

surgery, the surgeon enters the comment at the keyboard or the like. The comment is, 

as shown in FIG. 5, displayed in the comment display area 54. 

[0111] When a reply to the comment is requested by the remotely supporting 

surgeon, a reply is sent to the operating room 5. The surgeon in the operating room 5 

can acknowledge the reply by looking at the second monitor 29. 

[01 12] The remotely supporting surgeon can record an endoscopic image 

captured by the video capture control unit 44. When a recording instruction is given 

at the keyboard 34, a still image is recorded in the HDD 41. 

[0113] Selected reduced images made by reducing in size any of endoscopic 

images accumulated in the HOD 41 can be displayed in the thumbnail display area 

51 shown in FIG. 5. 

[0114] In addition to the endoscopic images, radiographic images of the patient 
7 can be transmitted from the first controller 24 to the second controller 32 in the 
remote control room 6. Reduced images of any of radiographic images accumulated 
in the HOD 41 can be displayed. 

[01 15] For performing surgery on the patient 7, the remotely supporting surgeon 
can make proper diagnosis by referencing the images on the display device 33. The 
remotely supporting surgeon can provide the operating room 5 with support 
information helpful in the surgery according to the diagnosis. 
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(01 16] Image information displayed as an overlay on the display device 33 is 
displayed on the second monitor 29 via the signal transmission apparatuses 28 and 
27. The surgeon in the operating room 5 carries out a surgical procedure with 
reference to the contents of the display on the second monitor 29 including the 
support information. The surgeon in the operating room S can thus accomplish the 
surgical procedure properly. 

[0117] According to the present embodiment, as described in the flowchart of 
FIG. 6, surgical information acquired in the operating room 5 can be displayed in the 
remote control room 6. 

[0118] Specifically, the power supplies of the systems 2 and 3 are turned on. At 
step SI, patient data of the patient 7 undergoing surgery and control information used 
to control the surgical instrument 20 are input to the controller 24. At step S2, 
endoscopic images having the patient data superimposed thereon are displayed on the 
first monitor 22. 

[0119] At step S3, the endoscopic images having the patient data superimposed 
thereon are transmitted from the CCU 24 to the signal transmission apparatus 28 in 
the remote control room 6 via the signal transmission apparatus 27 over the public 
line 4. The patient data and control information input to the controller 24 are 
transmitted to the signal transmission apparatus 28 in the remote control room 6 via 
the signal transmission apparatus 27 over the public line 4. 

[0120] At step S4, the endoscopic images having the patient data superimposed 
thereon are displayed on the third monitor 31. The endoscopic images, patient data, 
and control signal are input to the controller 32. 

[0121] The keyboard 34 is used to initiate capture of a still image. A still image 
of any of the endoscopic images input to the controller 32 is obtained by the video 
capture control unit 44, and displayed on the display device 33, as shown in FIG. 5 
(step S6). 
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[0122] The patient data and the contents of control given to the surgical 
instrument 20 are displayed on the display device at the same time. 
[0123] At step S7 if the controller 32 determines whether data has been input, if 
the remotely supporting surgeon enters support data at the keyboard 34 or the like, 
the data is overlaid (step S8). The data-overlaid image displayed on the display 
device 33 is transmitted to the operating room 5 (step S9), and then displayed on the 
second monitor 29 (step S 1 0). 

[0124] According to the present embodiment, the remotely supporting surgeon 
can view images showing a surgical scene nearly in real time on the third monitor 31. 
By looking at the display device 33, the remotely supporting surgeon can check the 
state of the surgical instrument 20 in the operating room 5 nearly in real time at any 
time, and also check patient information. For ordinary and emergency surgery, the 
remotely supporting surgeon can ascertain the state of the surgical instrument 20 and 
the patient information at any time. The remotely supporting surgeon can therefore 
quickly provide the surgeon in the operating room 5 with support information helpful 
in carrying out a proper surgical procedure. The surgeon in the operating room 5 can 
reference the support information and carry out a surgical procedure smoothly and 
quickly however complicated the surgical procedure is. 

[0125] The optical endoscope 9 is not limited to a system of relay lenses to 
propagate an optical image. An optical endoscope adopting an image guide in which 
a fiber bundle is used to propagate an optical image may be used. The surgeon in the 
operating room 5 and the remotely supporting surgeon in the remote control room 6 
can reciprocally transfer a voice signal. 

[0126] The second embodiment of the present invention will be described with 
reference to FIG. 7. Compared with the first embodiment, the present embodiment 
adopts an electronic endoscope 62 with a built-in imaging device instead of the 
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optical endoscope 9 and TV camera 1 5 of the endoscopic imaging apparatus 10 in the 
surgery system 2. 

[0127] The electronic endoscope 62 has an insertion unit 63, an operation unit 

64 formed at the back end of the insertion unit 63, and a universal cord 65 extending 
from the operation unit 64. A connector 66 attached to the end of the universal cord 

65 is plugged into the light source apparatus 17, whereby light is supplied from the 
light source apparatus 17. 

[0128] An imaging device (not shown) is incorporated in the distal part of the 
insertion unit 63. The imaging device is connected to the CCU 21 over a signal line 
67 linked to the connector 66. 

[0129] A video signal output to the second monitor 29 is also input to an image 
recording apparatus 69. When a recording instruction is input to the controller 24 
using the touch panel 25, an image desired to be recorded can be recorded. 
[0130] In the operating room 5, information, including the support information 
given by the remotely supporting surgeon during surgery, may be referenced as 
material for cure after surgery or utilized as significant material for future surgery. 
[0131] According to the present embodiment, a local area network (LAN) 70 is 
adopted as a communication line linking the surgery system 2 and the remote support 
system 3 employed in the first embodiment in place of the public line 4. The other 
components are identical to those of the first embodiment. Operations to be executed 
by the present embodiment are nearly identical to those by the first embodiment. 
[0132] When a remotely supporting surgeon needs radiographic images or the 
like in addition to endoscopic images so as to evaluate a patient's condition, the 
image information may be transferred from the controller 24 in the operating room 5 
to the controller (HDD 41 therein) in the remote control room 6. 
[0133] The third embodiment of the present invention will be described with 
reference to FIG. 8. Compared with the endoscope system 1 of the first embodiment, 
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a system of the present embodiment does not have the third monitor 3 1 installed in 
the remote control room 6 but uses the display device 33 to display the contents of 
display on the third monitor 3 1 . 

[0134] FIG. 8 shows a practical example of the contents of a display on the 
display screen 33a of the display device 33 in the present embodiment. 
[0135] As shown in FIG. 8, the display screen 33a of the display device 33 has a 
motion picture display area 61, a comment display area 62, a still image display area 
63, a state-of-surgical instrument display area 64, a patient information display area 
65, an endoscopic information display area 66, and a thumbnail image display area 
67. The motion picture display area 61 is used to display an endoscopic motion 
image. The comment display area 62 is used to display character information. The 
still image display area 63 is used to display a captured still image. The 
state-of-surgical instrument display area 64 is used to display the state of a surgical 
instrument. The patient information display area 65 is used to display patient 
information. The endoscopic information display area 66 is used to display 
information concerning an endoscope, for example, the state of the light source 
apparatus set for illumination or the state of the imaging device set for signal 
processing. The thumbnail image display area 67 is used to display thumbnail 
images. 

[0136] According to the present embodiment, for example, the first monitor 22 
has the motion picture display area 61, state-of-surgical instrument display area 64, 
patient information display area 65, and endoscopic information display area 66 that 
are shown in FIG. 8. These areas are used to display a motion picture of endoscopic 
Images, the state of a surgical instrument, patient information, and endoscopic 
information, respectively. 

[0137] On the first monitor 22, endoscopic images carried with a signal 
processed by the CCU 21 are displayed in the motion picture display area 61. The 

00622905.1 



-22- 



state of a surgical instrument set using the controller 24 is displayed in the 
state-of-surgical instrument display area 64. Patient information input from the 
magnetic card reader 26 via the controller 24 is displayed in the patient information 
display area 65. Endoscopic information sent from the controller 24 is displayed in 
the endoscopic information display area 66. The endoscopic information includes the 
tone setting and others for signal processing performed by the CCU 21, and the light 
adjustment setting for adjusting light emanating from the light source apparatus. 
[0138] The contents of display on the first monitor 22 are input from the CCU 
21 to the controller 32 via the signal transmission apparatus 27 and the signal 
transmission apparatus 28 in the remote control room 6. The same contents displayed 
on the first monitor 22 are displayed on the display device 33. 
[0139] When the surgeon in the operating room 5 has a comment that must be 
sent to the remote control room 6, the surgeon enters the comment at the keyboard 
29. The comment is then sent from the controller 24 to the controller 32 via the 
signal processing apparatus 27 and the signal processing apparatus in the remote 
control room 6. The contents of the comment are displayed in the comment display 
area 62 on the display device 33. 

[0140] In the remote control room 6, when a motion picture displayed on the 
display device 33 contains an image that must be captured, a capture instruction is 
entered at the keyboard 34 connected to the controller 32. The image is then captured 
as a still image, and the still image is displayed in the still image display area 62. 
[0141] A mark indicating a region to be resected may be appended to the still 
image displayed in the still image display area 63. 

[0142] For displaying a thumbnail image of any of images accumulated in the 
HDD 41, a thumbnail image display instruction is entered. The thumbnail image is 
then displayed in the thumbnail image display area 63. 
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[0143] A reply comment may be entered relative to a comment displayed in the 
comment display area 67. A comment, such as a note explaining to what should be 
paid attention during resection, may be entered. 

[0144] Among the contents of the display on the display device 33, image 
information, including a still image displayed in the still image display area 62 and 
the contents of a comment displayed in the comment display area 67, are sent to the 
controller 32 via the signal transmission apparatus 28 and the signal transmission 
apparatus 27 in the operating room 5. The image information is then displayed on the 
second monitor 29. 

[0145] The surgeon in the operating room 5 looks at the second monitor 29 
displaying the support information sent from the remote control room 6, and carries 
out a surgical procedure such as resection. 

[0146] When a plurality of surgical instruments is employed, the states of the 
plurality of surgical instruments A and B, for example, are, as shown in FIG. 8, 
displayed in the state-of-surgical instrument display area 64. 

[0147] According to the present embodiment, an endoscopic image, the state of 
a surgical instrument, patient information, and endoscopic information that are the 
same as those displayed in the operating room 5 can be displayed on the display 
device 33 in the remote control room 6 all of the time. The settings or controlled 
states of almost all electric equipment employed in a surgical procedure performed 
under endoscopic observation in the operating room 5 can be checked in the remote 
control room 6 all of the time. 

[0148] For example, the setting of a surgical instrument may be modified using 
the touch panel 25. Modification information is sent from the controller 24 to the 
CCU 21. The state of the surgical instrument displayed in the state-of-surgical 
instrument display area 62 on the first monitor 22 is then modified and displayed. 
The set state of the surgical instrument appended to image information sent from the 
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CCU 21 to the remote control room 6 via the signal transmission apparatus 27 is also 
modified. The state of the surgical instrument displayed in the state-of-surgical 
instrument display area 64 on the display device 33 is also modified. In the remote 
control room 6, information set in the operating room 5 is available nearly in real 
time without the necessity of performing any manipulation for acquiring needed 
information. 

[0149] The surgeon in the operating room 5 can carry out a surgical procedure 
while taking account of information displayed on the second monitor 29 and viewing 
endoscopic images displayed on the first monitor 22. 

[0150] The present embodiment has nearly the same advantages as the first 
embodiment. 

[0151] Patient information may be displayed in the comment display area 62. 
Endoscopic information may not be displayed on the first monitor 22 but may be sent 
from the controller 24 to the remote control room 6. In the remote control room 6, 
information items to be displayed may be selected. 

[0152] A motion picture, the state of a surgical instrument, and patient 
information displayed on the first monitor 2 may be sent to the remote control room 
6 and displayed on, for example, the third monitor 31. The other items of support 
information, that is, a still image and others, may be displayed on the display device 
33. 

[0153] Specifically, endoscopic images, the state of a surgical instrument, and 
patient information that are the same as those displayed on the first monitor 22 in the 
operating room 5 may be displayed in the remote control room 6. In the remote 
control room 6, when more support information must be appended to these 
information items, a still image, containing information such as a mark, and 
thumbnail images displayed on the display device 33 are referenced. Then, a support 
image, such as a still image having a mark appended thereto, or support information 
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including a comment may be sent to the operating room 5 and displayed on the 
second monitor 29. 

[0154] Depending on whether the operating room 5 and remote control room 6 
are linked by the public line 4, control of the display can be simplified. In either case, 
a display format according to which information is displayed on the first monitor 22 
remains unchanged. The surgeon in the operating room 5 will therefore be able to 
carry out a surgical procedure smoothly while referencing data displayed in the same 
display format. 

[0155] In the remote control room 6, endoscopic images, the state of a 
surgical-instrument, and patient information, which are arranged in the same display 
format as they are on the first monitor 22 in the operating room 5, can always be 
referenced on the third monitor 3 1 . 

[0156] In the above embodiments, the remotely supporting surgeon may be 
permitted to set an output value of electric energy supplied to an electric cautery 
instrument or enter control information used to control the movement of a surgical 
instrument. Specifically, in cases where the remotely supporting surgeon can set a 
more proper value more readily, the remotely supporting surgeon may be permitted 
to send a control signal from the controller 32 to the controller 24 in the operating 
room 5 using the keyboard 34 or touch panel. The movement of the surgical 
instrument 20 may be controlled based on the control signal. 
[0157] Endoscopic images output from the CCU 21 may be displayed on the 
first monitor 22. Control information used to control a treatment instrument and 
patient information also may be input from the controller 2 and superimposed on the 
endoscopic images. 

[0158] The fourth embodiment of the present invention will be described with 
reference to FIG. 9 and FIG. 10. 
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[0159] A remote surgery support system 7 1 in accordance with the fourth 
embodiment of the present invention, shown in FIG. 9 and FIG. 10, has a surgery 
system 72 and a remote support system 73 linked by a public line 72, such as the 
integrated services digital network (ISDN). The remote support system 73 is located 
remotely to the surgery system 72. The surgery system 72 and remote support system 
73 are installed in an operating room 75 and a remote room or a remote control room 
76, respectively. 

[0160] The surgery system 72 installed in the operating room 75 has an 
endoscopic imaging apparatus 78 to observe a patient 77 with an endoscope and a 
surgical instrument 79 to perform surgery under endoscopic observation. The 
surgical instrument 79 may be an electric cautery instrument, a pneumoperitoneum 
unit, or an ultrasonic surgical instrument. 

[0161] The endoscopic imaging apparatus 78 and surgical instrument 79 are 
connected to a first controller 80 for controlling the surgical instrument 79 and the 
first controller 80. 

[0162] To be more specific, the surgical instrument 79 has, as shown in FIG. 10, 
a surgical instrument body 81, such as an electric cautery instrument, and a power 
supply unit 82 for supplying operating power to the surgical instrument body 81. The 
surgical instrument body 81 is advanced into the abdominal cavity of the patient 77 
to perform surgery. 

[0163] The endoscopic imaging apparatus 78 has an optical endoscope 83 to 
observe a surgical scene. A TV camera 85 with a built-in solid-state imaging device, 
for example, a built-in charge couple device (CCD) 84 is mounted on the endoscope 
83, providing an endoscopic imaging means for producing endoscopic images. 
[0164] According to the present embodiment, the endoscope 83 is, for example, 
a rigid endoscope having a rigid insertion unit 86. The TV camera 85 is mounted on 
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an eyepiece unit 87 formed at the back end of the insertion unit 86 so that the TV 
camera can be dismounted freely. 

[0165] The endoscope 83 is connected to a light source apparatus 88 with a light 
guide cable 89. Light emanating from a lamp (not shown) in the light source 
apparatus 88 is propagated over a light guide contained in the light guide cable 89 
and a light guide running through the endoscope 83. The propagated light is emitted 
from the distal end surfaces of the light guides fixed in an illumination window 
formed at the distal end of the insertion unit 87. An object, such as an intracavitary 
organ, is thus illuminated. 

[0166] An objective lens (not shown) is fixed in an observation window 
adjoining the illumination window, and forms an optical image of the object. The 
optical image is propagated backwardly along an optical image propagating means 
incorporated in the insertion unit 84, for example, a system of relay lenses. 
Consequently, the image of the object can be viewed in enlarged proportions through 
an eyepiece (not shown) in the eyepiece unit 87 formed at the back end of a 
hand-held portion. The optical image propagating means incorporated in the 
endoscope 83 may be an image guide formed with a fiber bundle rather than the 
system of relay lenses. 

[0167] A propagated optical image is projected on the CCD 84 via an image 
formation lens 90 of the TV camera 85. The TV camera 85 is mounted on the 
eyepiece unit 87 so that the TV camera can be dismounted freely. The CCD 84 is 
connected to a camera control unit (CCU) 91 with a signal cable. A signal 
photoelectrically converted by the COD 84 is processed to produce a standard video 
signal. The video signal is then output to the first monitor 92. Consequently, 
endoscopic images representing an intracavitary organ and the distal part of the 
surgical instrument body 1 1 used to perform surgery on the organ projected on the 
CCD 84 are displayed on the first monitor 92. 
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[0168] The CCU 91 and power supply unit 82 are connected to a controller 80 
for controlling the CCU 91 and the power supply unit 82. 

[0169] The controller 80 is also connected to a touch panel 95 or the like used 
to enter an instruction for controlling operations, and a magnetic card reader or the 
like (not shown) for inputting patient data. 

[0170] The touch panel 95 may be used to control the CCU 91 via the controller 

80 so as to change tones or the like. The movement of the surgical instrument body 

81 of the surgical instrument 79 may be controlled using the touch panel 95. When 
the surgical instrument body 81 is, for example, an electric cautery instrument, the 
output level of electric energy supplied from the power supply 82 to the electric 
cautery instrument for the purpose of incision or coagulation can be set using the 
touch panel 95. 

[0171] According to the present embodiment, the CCU 91 of the endoscopic 
imaging apparatus 78 is connected to a first image transmission apparatus 97 serving 
as a signal transmitting means via a signal isolating means, for example, a video 
transformer 96. Specifically, endoscopic images produced by the endoscopic imaging 
apparatus 78 are converted into a video signal using the CCU 91 . The video signal is 
applied to the primary winding of the video transformer 96 enabling transmission of 
a signal having frequency components in a wide frequency band. The secondary 
winding of the video transformer 96 is connected to the image transmission 
apparatus 97. Thus, the video signal is supplied to the image transmission apparatus 
97 with electrical isolation between the CCU 91 and image transmission apparatus 
97 maintained. 

[0172] The image transmission apparatus 97 converts the video signal, which is 
input from the CCU 91, into a signal transmissible over the public line 74, so that the 
signal can be transmitted to a second image transmission apparatus 98 in the remote 
control room 76 over the public line 74. The image transmission apparatuses 97 and 
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98 can transmit an image signal having frequency components in a wide frequency 
band. A control signal or voice signal having lower frequency components can 
therefore be transmitted. 

[0173] The controller 80 in the operating room 75 is connected to the image 
transmission apparatus 97 via signal isolating means, for example, optical modems 
99. The optical modems 99 in the controller 80 and image transmission apparatus 97 
are linked by an optical cable. A converter for converting an electric signal into an 
optical signal is attached to the end of the optical cable coupled to the controller 80. 
A converter for converting the optical signal into an electric signal is attached to the 
end of the optical cable coupled to the image transmission apparatus 97. The 
controller 80 is connected to the image transmission apparatus 97 via the signal 
isolating means. 

[0174] The image transmission apparatus 97 converts a signal, which has been 
sent from the image transmission apparatus 98 in the remote control room 76 to the 
image transmission apparatus 97 over the public line 74, into a video signal. The 
video signal is output to the second monitor 102, which is isolated from the image 
transmission apparatus 97, via a second video transformer 101 connected to the 
image transmission apparatus 97. Image information sent from the image 
transmission apparatus 98 can be displayed on a second monitor 102. The second 
monitor 102 is connected to the image transmission apparatus 97 via the signal 
isolating means. 

[0175] A control signal used to control the surgical instrument 79 sent from the 
controller 80, or patient data read with a magnetic card reader input to the controller 
80, are input to the image transmission apparatus 97 via the optical modems 99. The 
image transmission apparatus 97 converts the input signal into a signal transmissible 
over the public line 74, and transmits the resultant signal to the image transmission 
apparatus 98 in the remote room 76 over the public line 74. 
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[01 76] According to the present embodiment, electric equipment are connected 
to the image transmission apparatus 97 in the operating room 75 via the signal 
isolating means. If the public line 74 is struck by lightning, the electric equipment 
connected to the image transmission apparatus 97 can be protected from being 
damaged. This eliminates the possibility that a surgeon using the electric equipment 
to carry out a surgical procedure and the patient 77 undergoing surgery performed 
using the treatment instrument 79 may experience an electric shock. Thus, safety is 
ensured. 

[0177] Patient data read from a magnetic card using a magnetic card reader is 
input to the controller 80. The controller 80 may output the patient data to the CCU 
91 so that the patient data will be superimposed on endoscopic images. Otherwise, 
the controller 80 may output the contents of control to be imparted to the surgical 
instrument 79 to the CCU 91 so that the controlled or set state of the surgical 
instrument 79 will be superimposed on the endoscopic images. 
[0178] The remote support system 73 in the remote room 76 has a third monitor 
104 connected to the image transmission apparatus 98. Endoscopic images or the like 
sent from the operating room 75 are displayed on the third monitor 104. 
[0179] The image transmission apparatus 98 is connected to an indoor camera 
105. A video signal produced by the indoor camera 105 is output to the second 
monitor 102 via the image transmission apparatuses 98 and 97 over the public line 
74. The video signal carries support images of a remotely supporting surgeon 106, or 
material or an explanatory diagram indicating a region to be resected during surgery. 
The region to be resected is indicated by the remotely supporting surgeon 106. 
[0180] A voice microphone 107 is attached to the indoor camera 105. A voice 
signal originating from the remotely supporting surgeon 106 is therefore transmitted 
to the operating room 75, and reproduced with a loudspeaker (not shown) of the 
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second monitor 102. Thus, the remotely supporting surgeon 106 can give advice or 
the like to a surgeon (not shown) in the operating room 75 during surgery. 
[0181] Next, actions to be made in accordance with the present embodiment 
will be described. 

[0182] The image transmission apparatuses 97 and 98 are electrically connected 
over the public line 74. On receipt of a video signal representing endoscopic images 
sent from the CCU 91 or a control signal and patient data sent from the controller 77, 
the image transmission apparatus 97 converts the input signal into a signal 
transmissible over the public line 74. The resultant signal is transmitted to the image 
transmission apparatus 98 in the remote control room 76 over the public line 74. The 
image transmission apparatus 98 converts the received signal that has been 
transmitted over the public line 74, and outputs a resultant signal to the third monitor 
104. The endoscopic images and the control to be imparted to the surgical instrument 
79 and the patient data are displayed on the third monitor 104. 
[0183] The indoor camera 105 is connected to the image transmission apparatus 
98. The indoor camera 105 images the remotely supporting surgeon 106 in the 
remote control room 76 or an explanatory diagram used for surgery. The image 
transmission apparatus 98 converts the image signal into a signal transmissible over 
the public line 74, and transmits the resultant signal to the image transmission 
apparatus 97 in the operating room 75 over the public line 74. 
[0184] Images produced by the indoor camera 105 are displayed on the second 
monitor i02 via the image transmission apparatus 97 and video transformer 101. 
Verbal advice given by the remotely supporting surgeon 106 also is transmitted in the 
same manner, and reproduced with a loudspeaker (not shown) together with the 
images displayed on the display screen of the second monitor 102. 
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[01 85] The surgeon in the operating room 75 can carry out a surgical procedure 
properly owing to the images displayed on the second monitor 102 and support such 
as the advice received along with the images. 

[0186] Owing to the foregoing components and operations, for example, even if 
the public line 74 is struck by lightning, the first monitor 92, second monitor 102, 
CCU 91, and controller 80 can be protected from being damaged. This is because the 
electric equipment connected to the image transmission apparatus 97 in the operating 
room 75 are electrically isolated due to the video transformers 96 and 101 and the 
optical modems 99. Consequently, the possibility that an surgeon handling the 
electric equipment and the patient 77 connected to the electric equipment may 
experience an electric shock can be eliminated reliably. 

[0187] According to the present embodiment, even if the public line 74 is struck 
by lightning, since the electric equipment connected to the image transmission 
apparatus in the operating room 73 are electrically isolated from one another, the 
electric equipment can be protected from being damaged. Moreover, the possibility 
that the patient 77 and surgeon may experience an electric shock can be eliminated 
reliably. 

[0188] Photocouplers or the like may be adopted as the signal isolating means 
instead of the video transformers 96 and 101 . A video signal or the like may thus be 
transmitted. 

[0189] The endoscopic imaging apparatus 78 in accordance with the present 
embodiment has an endoscope with an external TV camera having the TV camera 85 
with the built-in CCD 84 mounted on the optical endoscope 83. The present 
invention can also be adapted for an electronic endoscope having the CCD 84 
incorporated in the distal part of an insertion unit thereof. 

[0190] The fifth embodiment of the present invention will be described with 
reference to FIG. 1 1 . FIG. 1 1 shows the configuration of a remote surgery support 
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system 111 in accordance with the fifth embodiment of the present invention. The 
remote surgery support system 1 1 1 has the electric equipment connected to the image 
transmission apparatus 98 in the remote support system room 76, which is shown in 
FIG. 10, via signal isolating means. 

[0191] To be more specific, the image transmission apparatus 98 is connected to 
the third monitor 104 and indoor camera 105 via video transformers 112 and 113. 
The other components are identical to those of the fourth embodiment. The same 
reference numerals will be assigned to the components identical to those of the fourth 
embodiment. The description of the components will be omitted. 
[0192] Actions made in accordance with the present embodiment are nearly 
identical to those with the fourth embodiment. For example, if the public line 74 is 
struck by lightning, the electric equipment connected to the image transmission 
apparatus 87 in the surgery system 72 are protected from being damaged as they are 
in the fourth embodiment. Moreover, the possibility that the surgeon and patient 77 
may experience an electric shock is reliably eliminated as it is in the fourth 
embodiment. In addition, the electric equipment connected to the image transmission 
apparatus 98 in the remote control room 73, or more particularly, the third monitor 
104 and indoor camera 105 can be protected from being damaged. 
[0193] According to the present embodiment, the electric equipment connected 
to the image transmission apparatus 97 in the surgery system 72 can be protected 
from being damaged. The possibility that the surgeon and patient 77 may experience 
an electric shock can be eliminated reliably. The electric equipment connected to the 
image transmission apparatus 98 in the remote support system 73 also can be 
protected from being damaged. 

[0194] When the remotely supporting surgeon 106 handles the electric 
equipment connected to the image transmission apparatus 93, the possibility that the 
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remotely supporting surgeon 1 06 may experience an electric shock can be eliminated 
reliably. 

[0195] The sixth embodiment of the present invention will be described with 
reference to FIG. 12. FIG. 12 shows the configuration of a remote surgery support 
system 121 in accordance with the sixth embodiment of the present invention. 
[0196] Instead of connecting the controller 80 and the image transmission 
apparatus 97 via the optical modems 99, as shown in FIG. 10, the remote surgery 
support system 121 has the controller 80 connected to a first infrared transmitter 
receiver or IR transmitter receiver 122, in FIG. 12. The image transmission 
apparatus 97 is connected to a second infrared transmitter receiver 123. The first and 
second infrared transmitter receivers 122 and 123 communicate with each other in a 
wireless state using infrared light. Thus, image information or the like can be 
communicated with the electrical isolation between the controller 80 and image 
transmission apparatus 97 maintained. 

[0197] The other components are identical to those of the fifth embodiment. 
[0198] Operations to be executed by the present embodiment will be described. 
The present embodiment is different from the fifth embodiment in terms of an 
operation to be executed by a unit for transmitting a control signal or the like sent 
from the controller 80 to the image transmission apparatus 98. The unit will be 
described below. 

[0199] An electric signal, such as a control signal sent from the controller 80, is 
converted into an infrared signal by the infrared transmitter receiver 122, and 
transferred to the infrared transmitter receiver 123. The infrared transmitter receiver 
123 receives the infrared signal, converts it into an electric signal, and supplies the 
electric signal to the image transmission apparatus 97. The image transmission 
apparatus 97 transmits the signal to the image transmission apparatus 98 over the 
public line 74 The image transmission apparatus 98 outputs the signal to the third 
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monitor 104 via the connected video transformer 112. The signal is then 
superimposed on a video signal carrying endoscopic images sent from the CCU 91. 
[0200] If the public line 74 is struck by lightning, almost the same operation as 
that by the fifth embodiment is expected. Namely, the electric equipment connected 
to the image transmission apparatus 97 in the surgery system 72 can be protected 
from being damaged, and the possibility that the surgeon and patient 77 may 
experience an electric shock can be eliminated reliably. The electric equipment 
connected to the image transmission apparatus 98 in the remote support system 73 
also is protected from being damaged. 

[0201] According to the present embodiment, the controller 80 and image 
transmission apparatus 97 communicate with each other in a wireless manner using 
infrared light. Alternatively, the infrared communication means may be substituted 
for the video transformer 86 or 101 connected between the image transmission 
apparatus 97 and CCU 91 or between the image transmission apparatus 97 and 
second monitor 102. 

[0202] The present embodiment has nearly the same advantages as the fifth 
embodiment. Namely, the electric equipment connected to the image transmission 
apparatus 97 in the surgery system 72 can be protected from being damaged, and the 
possibility that the surgeon and patient 77 may experience an electric shock can be 
eliminated reliably. The electric equipment connected to the image transmission 
apparatus 98 in the remote support system 73 also is protected from being damaged. 
[0203] When the remotely supporting surgeon 106 handles the electric 
equipment connected to the image transmission apparatus 98, the possibility that the 
remotely supporting surgeon 106 experiences an electric shock can be eliminated 
reliably. 

[0204] An apparatus designed for infrared wireless communication may be 
substituted for the video transformer 1 12 or 1 13 in the remote room 76. 



-36- 



[0205] The seventh embodiment of the present invention will be described with 
reference to FIG. 13. FIG. 13 shows the configuration of a remote surgery support 
system 131 in accordance with the seventh embodiment of the present invention. 
[0206] According to the present embodiment, the remotely supporting surgeon 
106 in the remote room 76 shown in FIG. 12 can operate the controller 94 in the 
operating room 75 so as to send a control signal used to control the surgical 
instrument 79. Specifically, a touch panel 132 or a keyboard is an input means, and a 
controller 133 is included for producing a control signal according to an entry made 
by manipulating the touch panel 132. The controller 133 is connected to the image 
transmission apparatus 98 via optical modems 134 serving as signal isolating means. 
The other components are identical to those of the sixth embodiment. 
[0207] One of operations to be executed by the present embodiment is such that 
the remotely supporting surgeon 106 can control the surgical instrument 79 or can set 
parameter values for the surgical instrument 79. In other words, depending on the 
surgery, the remotely supporting surgeon 106 maybe able to set more appropriate 
parameter values, including an output value of electric energy supplied to, for 
example, an electric cautery instrument used for resection. The present embodiment 
would prove useful in this kind of case. 

[0208] The other operations are nearly identical to these of the sixth 
embodiment. 

[0209] The present embodiment has nearly the same advantages as the sixth 
embodiment. 

[0210] The wireless communication means is not limited to the infrared 
communication means but may use a high-frequency electromagnetic wave or the 
like. 

[021 1] The controller 1 33 in the remote room 76 has the ability to transmit a 
control signal, which is produced according to an entry made at the touch panel 132, 
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to the operating room 75 via the optical modems 134 and image transmission 
apparatus 98. A video signal of endoscopic images or a control signal transmitted 
from the CCU 91 or controller 80 in the operating room 75 may be input to the 
controller 133 via the image transmission apparatus 98 and optical modems 134. A 
video capture control unit for capturing a video signal or an I/O interface in the 
controller 133 captures the video signal. The position of a region to be resected 
during surgery may then be marked in images carried by the video signal. The video 
signal may then be returned to the operating room. 

[0212] A display device, such as a monitor, may be connected to the controller 
133 so that images captured by the video capture control unit in the controller 133 or 
information indicated with a control signal can be displayed as an overlay. 
[0213] A communication line linking the operating room and remote room may 
be, aside from the public line 74, a local area network (LAN) or a wide area network 
(WAN). 

[0214] A medical information processing system will be described with 
reference to FIG. 14 to FIG. 18. The medical information processing system is 
intended to improve the efficiency of retrieving medical information, including a 
plurality of different kinds of image data that have relation to one another, so as to 
thus improve the efficiency of working on information at a medical institute or the 
like. 

[0215] As shown in FIG. 14, a medical information processing system consists 
mainly of an operating room system 201, a plurality of examination room systems 
202, an information accumulation system 203, and an information reference 
apparatus 204. The operating room system 201 is installed in an operating room or 
any other site and acquires endoscopic image data, surgery system information, and 
patient information. The examination room apparatuses 202 are installed in 
examination rooms dedicated to examinations performed using an MRI system, a CT 
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scanner, a diagnostic ultrasound system, and an X-ray system and acquire a plurality 
of kinds of image data different from endoscopic image data. The information 
accumulation system 203 is installed in a site other than the operating room and the 
examination rooms, accumulates various kinds of image data acquired by the 
operating room system 201 and the examination room systems 202 and non-image 
data, and produces clinical recordings. The information reference apparatus 204 is 
used to reference medical information accumulated in the information accumulation 
system 203. The operating room system 201 and the information accumulation 
system 203, the examination room systems 202 and the information accumulation 
system 203, and the information reference apparatus 204 and the information 
accumulation system 203 are linked by public lines or any other communication lines 
205, 206, and 207, respectively. 

[02161 The operating room system 201 consists mainly of an endoscope system 
21 1, a display device 212, a surgical apparatus 213, a control apparatus 214, a 
manipulating means, such as an operator panel 215, a patient information input 
means, such as a magnetic card reader 216, a signal transmission apparatus 217, and 
a display device 218. The endoscope system 211 images a region to be examined in a 
patients body cavity and acquires endoscopic image data. The display device 212 
displays endoscopic images produced by the endoscope system 211. The surgical 
apparatus 213 is, for example, an electric cautery instrument, a pneumoperitoneum 
unit, or an ultrasonic surgical apparatus. The control apparatus 214 has the ability to 
acquire surgical apparatus information, such as measurement information provided 
by the surgical apparatus 213. The operator panel 215 is connected to the control 
apparatus 214 to input a control instruction or the like, according to which the 
surgical apparatus 213 should be controlled, to the control apparatus 214. The 
magnetic card reader 216 reads patient information from a magnetic card and 
supplies it to the control apparatus 214. The signal transmission apparatus 217 
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transmits endoscopic image data, surgical apparatus information, and patient 
information to the information accumulation system 203 installed in another site over 
a communication line 205, and communicates with the information accumulation 
system 203 so as to receive data. The display device 218 displays surgical apparatus 
information and patient information supplied from the control apparatus 214 and 
information received from the other sites via the signal transmission apparatus 217. 
[0217] The examination room systems 202 include various examination systems 
(not shown), image filing apparatuses 221, and a communicating means, such as a 
modem 222. The examination systems include an MRI system, a CT scanner, a 
diagnostic ultrasound system, and an X-ray system. The image filing apparatuses 221 
accumulate image data acquired by the examination systems. The modem 222 
connects the image filing apparatuses 221 to the other sites over communication lines 
206. The image data accumulated in the image filing apparatuses 221 can be 
transmitted to the other sites. The image data stored in the image-filing apparatuses 
221 may not be solely image data but may contain non-image data, such as data 
associated with the image data, for example, findings. 

[0218] The information accumulation system 203 consists mainly of a signal 
transmission apparatus 231, a display device 232, an information accumulation 
apparatus 233, a display device 234, a printer 235, and a communicating means, such 
as a modem 236. The signal transmission apparatus 231 communicates with the 
signal transmission apparatus 217. The display device 232 displays endoscopic image 
data, surgical apparatus information, and patient information received from the other 
sites via the signal transmission apparatus 231. The information accumulation 
apparatus 233 accumulates medical information, such as the endoscopic image data, 
surgical apparatus information, and patient information received from the other sites 
via the signal transmission apparatus 231, and image data other than the endoscopic 
image data. The information accumulation apparatus also edits the medical 
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information. The display device 234 displays the medical information accumulated 
in the information accumulation apparatus 233. The printer 235 prints clinical 
records of edited medical information accumulated in the information accumulation 
apparatus 233. The modem 236 enables transfer of the medical information between 
the examination room systems 202 or information reference apparatus 204 and the 
information accumulation apparatus 233. 

[0219] The information reference apparatus 204 consists of a communicating 
means, such as a modem 241, and a terminal unit 242. The modem 241 enables 
transfer of medical information to or from the other sites. The terminal unit 242 is 
used to reference the medical information accumulated in the information 
accumulation system 203 via the modem 241 . The terminal unit 242 may be 
designed, if necessary to edit the medical information accumulated in the information 
accumulation system 203. 

[0220] As shown in FIG. 15, the information accumulation apparatus 233 
consists mainly of a control circuit 251, a video signal input/output circuit 252, a 
communication interface 253, a display control circuit 254, and a printing control 
circuit 255. The control circuit 251 controls the components of the information 
accumulation apparatus 233. The video signal input/output circuit 252 inputs to or 
outputs from the signal transmission apparatus 231 endoscopic image data. The 
communication interface 253 enables input or output of medical information between 
the signal transmission apparatus 231 and modem 236. The display control circuit 
254 is controlled by the control circuit 251 and outputs a video signal used to drive 
the display device 234. The printing control circuit 255 is controlled by the control 
circuit 251 and outputs a video signal used to drive the printer 235. 
[0221] The control circuit 251 has such hardware devices as a processor 251a, a 
memory 251b, and a hard disk 251c. The processor 251a is operated by software. 
Software programs to be run by the processor 251a and medical information are 



-41 - 



stored in the memory 25 lb and hard disk 251c. A work area to be used by the 
processor 251a is preserved in the memory 251b. 

[0222] The video signal input/output circuit 252 consists of a controller 252a, 
an encoder decoder 252b, an A/D converter 252c, and a D/A converter 252b. The 
controller 252a controls the circuit elements of the video signal input/output circuit 
252. the encoder decoder 252b encodes or decodes image data, the A/D/ converter 
252c converts input image data fro an analog form to a digital form. The D/A 
converter 252b converts the image data from the digital form to the analog form. 
[0223] As shown in FIG. 16, the control circuit 251 has such functional 
elements as a data memory 261, a data saver 262, a data retriever 263, a screen editor 
264, and a clinical record editor 265. The data memory 261 is used to store data. 
The data saver 262 saves data, which is input via the video signal input-output circuit 
252 and communication interface 253, in the data memory 261 . The data retriever 
263 retrieves and reads data from the data memory 261. The screen editor 264 edits 
data read by the data retriever 263 according to a screen display format. The clinical 
record editor 265 edits data read by the data retriever 263 according to a printout 
format of clinical records. 

[0224] As shown in FIG. 1 7, a medical information display screen 271 
displayed with the display device 234 by means of the information accumulation 
apparatus 233 consists of a view image area 272, a thumbnail image area 273, a 
patient information area 274, a surgical apparatus information area 275, and an 
operation menu display area 276. View images, including endoscopic images, are 
displayed in the view image area 272. A plurality of thumbnail images produced by 
reducing image data, for example, thinning pixels constituting image data, is 
displayed in the thumbnail image area 273. Patient information is displayed in the 
patient information area 274. Surgical apparatus information is displayed in the 
surgical apparatus information area 275. The operation menu display area 276 is 
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used to execute such an operation as retrieval of medical information or display of 
images. A button 277 to initiate retrieval of medical information. 
[0225] Other buttons also are arranged in the operation menu display area 276. 
The buttons initiate other operations. When a button is selected using a mouse or the 
like (not shown), an allocated utility is activated. 

[0226] As shown in FIG. 18, a clinical record printout 281 output from the 
printer 235 driven by the information accumulation apparatus 233 has a view image 
area 282, a thumbnail image area 283, and a patient information area 284 arranged 
therein. 

[0227] Operations executed by the present embodiment will be described 
below. 

[0228] Prior to surgery, a magnetic card in which patient information is stored is 
inserted into the magnetic card reader 216. The patient information read by the 
magnetic card reader 216 is supplied to the control apparatus 214. When surgery 
starts, the endoscope system 211 and surgical apparatus 213 are employed. 
[0229] The endoscope system 21 1 is used to image a region to be observed in a 
patient f s body cavity. A video signal containing endoscopic image data produced by 
the endoscope system 21 1 is output to the display device 212. Endoscopic images 
are then displayed on the display device 212. The operator panel 215 may be 
manipulated to control the endoscope system 211, that is, to adjust the tone of the 
endoscopic images. The control apparatus 214 controls the endoscope system 211 
according to the manipulation. 

[0230] The operator panel 215 may be manipulated in order to control the 
surgical apparatus 213. The control apparatus 214 then controls the surgical 
apparatus 213 according to the manipulation. When the surgical apparatus 213 is 
controlled, power output to the electric cautery instrument or pressure applied to an 
abdominal cavity by a pneumoperitoneum unit is set to a certain level. Measurement 
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information provided by the surgical apparatus 213 is given to the control apparatus 
214. The measurement information pertains to, for example, pressure actually 
applied to a patient's abdominal cavity by the pneumoperitoneum unit. Thus, the 
patient information sent from the control apparatus 214 and surgical apparatus 
information are displayed on the display device 18. The surgical apparatus 
information contains information of the state of the endoscope system 211, 
information of the state of the surgical apparatus 213, and the measurement 
information provided by the surgical apparatus 213. 

[0231] Endoscopic image data produced by the endoscope system 211, and 
patient information and surgical apparatus information sent from the control 
apparatus 214 are supplied to the signal transmission apparatus 231 in the 
information accumulation system 203 over the communication line 205. The signal 
transmission apparatus 231 transfers the input endoscopic image data, patient 
information, and surgical apparatus information to the display device 232 connected 
to the signal transmission apparatus 231. The signal transmission apparatus 231 also 
transfers the data and information to the information accumulation apparatus 233. 
For transmitting image data, if necessary, the signal transmission apparatus 217 
encodes or decodes the image data. 

[0232] MRI data, CT image data, ultrasonic image data, and radiographic image 
data acquired in the examination rooms are accumulated in the image filing 
apparatuses 221 in the examination rooms. The image data accumulated in the image 
filing apparatuses 221 are transmitted to the information accumulation apparatus 233 
by way of the modems 222, communication lines 206, and modem 236. 
[0233] As mentioned above, medical information, composed of endoscopic 
image data, patient information, surgical apparatus information, and various kinds of 
image data other than the endoscopic image data, is transmitted to the information 
accumulation apparatus 233 via the signal transmission apparatus 231 and modem 
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236 serving as communicating means. The endoscopic image data sent to the 
information accumulation apparatus 233 via the signal transmission apparatus 231 
are transferred to the video signal input/output circuit 252. The endoscopic image 
data are converted into digital data or, if necessary, decoded, and then applied to the 
control circuit 251. 

[0234] The patient information, surgical apparatus information, and various 
kinds of image data sent to the information accumulation apparatus 233 via the signal 
transmission apparatus 231 and modem 236 are transferred to the control circuit 251 
via the communication interface 253. 

[0235] The medical information transferred to the control circuit 251 is saved in 
the data memory 261 by the data saver 262. The data saver 262 associates relevant 
medical information, for example, sorts medical information by patient name and 
then saves associated information in the data memory 261. For sorting and saving 
medical information in the data memory 261, for example, relevant medical 
information may be integrated into one file. 

[0236] For retrieving desired medical information from the data memory 261 , 
for example, the button 277 in the operation menu display area 276 in the medical 
information display screen 271 selected with a mouse or the like (not shown). A 
screen for prompting selection of keys relevant to retrieval (not shown) then appears. 
The keys relevant to retrieval are then pressed with the screen displayed. This causes 
the data retriever 263 to retrieve and read medical information designated with the 
pressed keys from the data memory 261 . 

[0237] For example, assuming that a patient name, a patient code, or the like for 
identifying a patient is keyed in, medical information concerning the designated 
patient is then read from the data memory 261 . Image data of the medical 
information may be, if necessary, converted into thumbnail image data, expressing 
reduced images, and output. 
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[0238] The screen editor 264 edits medical information retrieved by the data 
retriever 263 according to a screen display format. The resultant medical information 
is displayed on the display device 234 via the display control circuit 254 with 
respective information items allocated to the display areas of the medical information 
display screen 271. The thumbnail images produced by reducing in size image data, 
for example, are displayed in the thumbnail image area 273. Surgical apparatus 
information is displayed in the surgical apparatus information area 275, and patient 
information is displayed in the patient information area 274. 
[0239] In the thumbnail image area 273, not only endoscopic images, but also 
various kinds of images other than the endoscopic images are displayed. When any of 
one or more thumbnail images displayed in the thumbnail image area 273 is 
designated with a mouse or the like (not shown), a full-sized image associated with 
the designated thumbnail image appears in the view image area 272, In the view 
image area 272, not only an endoscopic image, but also any of the other various 
kinds of images can be displayed if necessary. 

[0240] The clinical record editor 265 edits medical information retrieved by the 
data retriever 263 according to a clinical record printout format. The resultant 
medical information is then supplied to the printing control circuit 255. 
Consequently, the printer 235 provides the clinical record printout 281. 
[0241] Medical information retrieved by the data retriever 263 is also output to 
the display device 218 by way of the video signal input/output circuit 252, 
communication interface 253, signal transmission apparatus 231, communication line 
205, and signal transmission apparatus 217. Image data output from the control 
circuit 251 are, if necessary, encoded by the video signal input/output circuit 252, 
converted into analog data, and then transmitted to the signal transmission apparatus 
23L 
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[0242] In response to a request for retrieval of medical information, the data 
retriever 263 retrieves and reads designated medical data from the data memory 261 
by way of the terminal unit 242, modem 241, communication line 207, modem 236, 
and communication interface 253. The medical data then are transmitted to the 
terminal unit 242. 

[0243] The information accumulation system 203 may be used as, for example, 
the remote support system 6 included in the first embodiment to support surgery. 
[0244] An operator of the information accumulation system 203 displays 
endoscopic image data and patient data, which are sent from the operating room 
system 201, on the display device 232 so as to monitor the progress of surgery. 
Information relevant to the surgery is displayed on the display device 234. According 
to the first embodiment, data stored in the remote support system 6 are utilized as 
information relevant to surgery. According to the present embodiment, any 
information can be retrieved from the examination room systems 202 or the like. 
[0245] Information useful in supporting an a surgeon, such as images 
representing what must be noted for surgery, is output to the display device 218 via 
the signal transmission apparatus 231 and the signal transmission apparatus 217 in 
the operating room system 201. 

[0246] As described above, according to the present embodiment, medical 
information, including a plurality of different kinds of correlated image data, is 
stored while being associated. The medical information, including the plurality of 
different kinds of correlated image data, can be retrieved easily. 
[0247] According to the present embodiment, the efficiency in retrieving 
medical information, including a plurality of different kinds of correlated image data, 
improves. This improves the efficiency in working on information at a medical 
institution. 
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[0248] The present embodiment is not limited to the aforesaid mode. Various 
variants of the present embodiment can be made without a departure from the spirit 
of the present invention. 

[0249] Specifically, the communication lines 205, 206, and 207 are not limited 
to public lines, but may be leased lines, local area networks (LAN), or any other 
communication lines. The information reference apparatus 204 may be installed in 
the same site as the operating room system 201 or the examination room systems 
202. The examination room systems 202 may be installed in the same site as the 
operating room system 201 or the information accumulation system 203. 
[0250] According to the present embodiment, an input means for inputting 
patient information is provided with the magnetic card reader 216. However, the 
input means is not limited to the magnetic card reader. Alternatively, an apparatus for 
reading patient information from an IC card, an optical card, or any other medium 
other than the magnetic card may be used. 

[0251] The image filing apparatus 221 may be provided with a computer having 
a display facility. 

[0252] The terminal unit 242 may be provided with a computer having a display 
utility. 

[0253] The information accumulation apparatus 233 may have the ability to 
display medical information read from the data memory 261 as well as medical 
information retrieved, if necessary, by way of the signal transmission apparatus 231 
and modem 236 on the medical information display screen 271 instead of 
accumulating the medical information. 

[0254] The modems 222, 236, and 241 may be replaced with any other 
communicating means, for example, terminal adaptors or routers. 
[0255] Retrieval-related keys to be pressed for retrieving medical information 
from the data memory 261 are not limited to keys identifying a patient, but may be 
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any other keys. For example, keys indicating a date of surgery, an operator, or 
identifying a kind of case may be used. A plurality of sets of keys may be used in 
combination. Related medical information is stored in the data memory 216 
according to the set of keys to be employed. 

[0256] According to the present embodiment, image data is transmitted from a 
remote site to the information accumulation system over a communication line. The 
present embodiment is not limited to this mode. Alternatively, a portable 
magneto-optical disk or the like on which image data is stored may be transported 
from a remote site to the information accumulation system. Image data may be saved 
from the recording medium to the information accumulation apparatus. 
[0257] A medical information processing system facilitating efficiency in 
producing clinical recordings by saving time required for acquiring and accumulating 
medical treatment information will be described with reference to FIG. 19 to FIG. 22. 
[0258] As shown in FIG. 19, a medical information processing system of the 
present embodiment consists mainly of an operating room system 301, an 
information accumulation system 302, and an information reference apparatus 303. 
The operating room system 301 is installed in an operating room or any other site, 
and acquires various kinds of medical information in the operating room or the like. 
The information accumulation system 302 is installed remotely from the operating 
room 301, accumulates various kinds of medical treatment information acquired by 
the operating room system 301, and produces clinical records. The information 
reference apparatus 303 is installed in a site different from the site in which the 
information accumulation system 302 is installed, and is used to reference clinical 
record information produced by the information accumulation system 302. The 
operating room system 301 and information accumulation system 302 and the 
information accumulation system 302 and information reference apparatus 303 are 
linked by communication lines 304 and 305, respectively. 
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[0259] The communication lines 304 and 305 may be public lines, leased lines, 
local area networks (LAN), or any other communication lines. The information 
reference apparatus 303 may be installed in the same site as the operating room 
system 301. 

[0260] The operating room system 301 consists mainly of an endoscope system 
3 1 1 , a display device 3 1 2, a surgical apparatus 3 1 3 , a control apparatus 3 1 4, a 
surgical apparatus control input means, such as a touch panel 315, patient 
information input means, such as a magnetic card reader 316, a signal transmission 
apparatus 317, and a display device 318. The endoscope system 311 images a region 
to be examined, such as a patient's intracavitary region, and produces a video signal 
containing endoscopic image data. The display device 312 displays an image 
corresponding to a video signal produced by the endoscope system 311. The surgical 
apparatus 313 is, for example, an electric cautery instrument, a pneumoperitoneum 
unit, or an ultrasonic surgical apparatus. The control apparatus 314 controls the 
surgical apparatus 313 and produces surgical apparatus information, including 
measurement information supplied from the surgical apparatus 313. The touch panel 
315 is connected to the control apparatus 314 and used to enter a control instruction 
or the like, according to which the surgical apparatus 313 is controlled, to the control 
apparatus 314. The magnetic card reader 316 reads patient information from a 
magnetic card and supplies it to the control apparatus 314. The signal transmission 
apparatus 317 transmits a video signal, surgical apparatus information, and patient 
information to the information accumulation system 302 located in another site over 
a communication line, and communicates with the information accumulation system 
302 so as to receive information therefrom. The video signal contains endoscopic 
image data produced by the endoscope system 311. The surgical apparatus 
information is supplied from the control apparatus 314. The display device 318 
displays the surgical apparatus information and patient information supplied from the 
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control apparatus 3 14 or information received from another site by the signal 
transmission apparatus 317. 

[0261] The information accumulation system 302 consists mainly of a signal 
transmission apparatus 321, a display device 322, a control apparatus 323, a display 
device 324, a printer 325, and a communicating means, such as a modem 326. The 
signal transmission apparatus 321 communicates with the signal transmission 
apparatus 317. The display device 322 displays endoscopic image information, 
surgical apparatus information, and patient information received from another site 
via the signal transmission apparatus 321. The control apparatus 323 accumulates 
medical treatment information, including the endoscopic image information, surgical 
apparatus information, and patient information received from another site via the 
signal transmission apparatus 321, and produces a video signal for generating a 
screen into which the images and information are integrated. The control apparatus 
323 also has the ability to integrate and edit the information so as to produce a 
clinical record, and to output the clinical record to the printer. The display device 324 
displays the information integrated by the control apparatus 323. The printer 325 
prints out the clinical record produced by the control apparatus 323. The modem 326 
helps transmit the information, which is integrated by the control apparatus 323, to 
the information reference apparatus 303 in another site over the communication line 
305. 

[0262] The information reference apparatus 303 consists mainly of a 
communicating means, such as a modem 331, and a terminal unit 332. The modem 
331 communicates with the communicating means, such as the modem 326. The 
terminal unit 332 communicates with the control apparatus 323 via the modem 331 
so as to reference information integrated by the control apparatus 323 or input or edit 
information. 
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[0263] As shown in FIG. 20, the control apparatus 323 consists mainly of a 
processor 351, a hard disk 352, a memory 353, a display control unit 354, a printing 
control unit 355, a communication interface 356, a video signal input/output circuit 
357, and a communication interface 358. The processor 351 controls the 
components of the control apparatus 323. Software programs to be run by the 
processor 351 and data, such as the endoscopic images and patient information, are 
stored in the hard disk 352 and memory 353. A work area used by the processor 35 1 
also is preserved in the memory 353. The display control unit and produces a video 
signal used to drive the display device 324. The printing control unit 355 produces a 
video signal used to drive the printer 325. The communication interface 356 helps 
transmit the patient information and surgical apparatus information to or from the 
signal transmission apparatus 321. The video signal input/output circuit 357 inputs to 
or outputs from the signal transmission apparatus 321 video information, including 
endoscopic image data. The communication interface 358 transmits information to 
the modem 326. 

[0264] The processor 351 consists mainly of an information accumulation unit 
351a, a display information production unit 351b, and a clinical record production 
unit 351c. The endoscopic image data, patient information, and surgical apparatus 
information, which are input independently via the signal transmission apparatus 
321, are accumulated in the information accumulation unit 351a with the software 
programs stored in the hard disk 352 and memory 353. The display information 
production unit 351b edits information accumulated in the information accumulation 
unit 351a, and produces screen information concerning a screen display on the display 
device 324. The clinical record production unit 351c edits the accumulated 
information and produces a clinical record to be output to the printer 325. 
[0265] The video signal input/output circuit 357 consists mainly of an A/D 
converter 357a, a D/A converter 357b, a video signal encoder decoder 357c, and a 
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controller 357d. The AID converter 357a converts an input analog video signal into a 
digital video signal. The D/A converter 357b converts a digital video signal into an 
analog video signal and outputs the analog video signal. The video signal encoder 
decoder 357c decodes a video signal input via the A/D converter 357a, samples a 
desired endoscopic image signal from the input video signal, encodes a video signal 
to be output, and supplies the resultant signal to the D/A converter 357b. The 
controller 357d controls the video signal encoder and decoder 357c and other 
elements of the video signal input/output circuit 357. 

[0266] As shown in FIG. 21, an endoscopic image area 362, a thumbnail image 
area 363, and a surgical apparatus information area 365 are defined in an integrated 
information screen 361 displayed on the display device 324 according to an output 
from the control apparatus 323. Endoscopic images are displayed in the endoscopic 
image area 362. A plurality of thumbnail images, reduced images produced by 
thinning pixels that constitute endoscopic image data, is displayed in the thumbnail 
image area 363. Patient information is displayed in the patient information area 364. 
Surgical apparatus information is displayed in the surgical apparatus information area 
365. 

[0267] As shown in FIG. 22, an endoscopic image area 372, a thumbnail image 
area 373, and a patient information area 374 are defined in a clinical record 371 
output from the printer 325 according to an output from the control apparatus 323. 
[0268] Operations executed by the present embodiment will be described 
below. 

[0269] A video signal containing endoscopic image data produced by the 
endoscope system 311 during endoscopic examination is transmitted to the signal 
transmission apparatus 317. The signal transmission apparatus 317 encodes the video 
signal and transmits it to the signal transmission apparatus 321 over the 
communication line 304. The video signal received by the signal transmission 
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apparatus 32 1 is converted into a digital signal by the A/D converter 357a in the 
video signal input/output circuit 357 of the control apparatus 323, and then decoded 
by the video signal encoder decoder 357c. 

[0270] If necessary, the video signal encoder decoder 357c samples desired 
endoscopic image data contained in the video signal under the control of the control 
unit 357d controlled by the processor 351 The endoscopic image data retrieved by 
the video signal input/output circuit 357 are accumulated in the information 
accumulation unit 351a. 

[0271] Surgical apparatus information sent from the touch panel 3 1 5 and 
surgical apparatus 313 to the control apparatus 314 is fed to the signal transmission 
apparatus 317. The signal transmission apparatus 317 transmits the surgical 
apparatus information to the signal transmission apparatus 21 over the 
communication line 304. The surgical apparatus information received by the signal 
transmission apparatus 321 is retrieved into the control apparatus 323 via the 
communication interface 356 and accumulated in the information accumulation unit 
351a. 

[0272] Patient information supplied from the magnetic card reader 316 to the 
control apparatus 314 is fed to the signal transmission apparatus 317. The signal 
transmission apparatus 317 transmits the patient information to the signal 
transmission apparatus 321 over the communication line 304. The patient 
information received by the signal transmission apparatus 321 is retrieved by the 
control apparatus 323 via the communication interface 356 and accumulated in the 
information accumulation unit 351a. 

[0273] As mentioned above, information, such as endoscopic image 
information, surgical apparatus information, and patient information, input from 
different apparatuses or developed on different occasions are accumulated in the 
information accumulation unit 351a in the control apparatus 323. 
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[0274] Information accumulation in the information accumulation unit 351a is 
edited based on a screen display format by the display information production unit 
351b. The integrated information screen 361 is displayed on the display device 325. 
Endoscopic image information, patient information, and surgical apparatus 
information input from different input units or developed on different occasions are 
integrated into the integrated information screen 361. In the integrated information 
screen 361, if any thumbnail image is selected from thumbnail images of endoscopic 
images displayed in the thumbnail image area 363, an endoscopic image associated 
with the selected thumbnail image may be displayed in the endoscopic image area 
362. 

[0275] Information accumulated in the information accumulation unit 35 1 a is 
edited based on a printout format by the clinical record production unit 351c. The 
printer 325 outputs a clinical record 371. Endoscopic image information and patient 
information input from different input units or developed on different occasions are 
integrated into the clinical record 371. 

[0276] Information accumulated in the control apparatus 323 is transmitted to 
the terminal unit 332 in the information reference apparatus 303. The information is 
transmitted by way of the modem 326 connected to the communication interface 358, 
the communication line 305 linked to the modem 326, and the modem 331 in the 
information reference apparatus 303. The terminal unit 332 uses, for example, a 
display device thereof to display information contained in the integrated information 
screen 361 or clinical record 371 in an integrated manner. The information reference 
apparatus 303 maybe installed in a plurality of sites. Information accumulated in the 
information accumulation system 302 may therefore be referenced or, if necessary, 
edited at any site. 

[0277] According to the present embodiment, medical treatment information, 
such as endoscopic image information, surgical apparatus information, and patient 
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information, that is input from different input units or develops on different 
occasions are accumulated in the control apparatus 323. The control apparatus 323 
causes the display device 324 to display the integrated information screen 361 into 
which the accumulated information is integrated and which contains clinical record 
information. The control apparatus 323 also causes the printer 325 to print out a 
clinical record. Consequently, the time required for acquiring and accumulating 
medical treatment information is reduced, and the efficiency in producing a clinical 
record improves. 

[0278] Not only endoscopic image information acquired by the endoscope 
system 311, but also information acquired by related instruments, such as various 
types of surgical instruments 313, can be accumulated. 

[0279] The magnetic card reader 3 16 is used to read patient information. The 
labor for entering patient information at a keyboard or the like is therefore 
eliminated. 

[0280] Medical treatment information can be referenced or edited in any site 
using the information reference apparatus 303. 

[0281] Medical treatment information acquired by the operating room system 
301 installed in an operating room is accumulated in the information accumulation 
system 302 installed in a remote site, for example, in another room. The medical 
treatment information can therefore be edited in another room. Tasks related to 
surgery can be processed to thus provide support from a remote site other than the 
operating room. If the information reference apparatus 303 is installed in the 
operating room, a clinical record produced with the support provided from the 
remote site can be referenced in the operating room. Consequently, work can be 
performed efficiently in the operating room. 



-56- 



[0282] The modems 326 and 331 may be replaced with any other 
communication units, for example, terminal adaptors. The hard disk 352 may be 
replaced with any other storage means, for example, a magneto-optical disk. 
[0283] The magnetic card reader 316 may be replaced with an apparatus for 
reading information from a portable medium other than a magnetic card, such as an 
IC card or an optical card. 

[0284] Although the present invention has been described in relation to 
particular embodiments thereof, many other variations and modifications and other 
uses will become apparent to those skilled in the art. It is preferred, therefore, that 
the present invention be limited not by the specific disclosure herein, but only by the 
appended claims. 
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